Bidirectional Helmholtz Machines
نویسندگان
چکیده
Efficient unsupervised training and inference in deep generative models remains a challenging problem. One basic approach, called Helmholtz machine or Variational Autoencoder, involves training a top-down directed generative model together with a bottom-up auxiliary model used for approximate inference. Recent results indicate that better generative models can be obtained with better approximate inference procedures. Instead of improving the inference procedure, we here propose a new model, the bidirectional Helmholtz machine, which guarantees that the top-down and bottom-up distributions can efficiently invert each other. We achieve this by interpreting both the top-down and the bottom-up directed models as approximate inference distributions and by defining the model distribution to be the geometric mean of these two. We present a lower-bound for the likelihood of this model and we show that optimizing this bound regularizes the model so that the Bhattacharyya distance between the bottom-up and top-down approximate distributions is minimized. This approach results in state of the art generative models which prefer significantly deeper architectures while it allows for orders of magnitude more efficient likelihood estimation.
منابع مشابه
Days on Diffraction' 2008 Helmholtz Solitons: Maxwell's Equations, Interfaces, Bistability & Counterpropagation
We give a brief overview of some new results in Helmholtz soliton theory. Firstly, fundamental considerations are made in terms of new contexts for Helmholtz solitons that arise directly from Maxwell’s equations. We then detail applications of Helmholtz solitons in material interface geometries: generalising Snell’s law to nonlinear beams and reporting new qualitative phenomena. Novel families ...
متن کاملBinocular Helmholtz Stereopsis
Helmholtz stereopsis has been introduced recently as a surface reconstruction technique that does not assume a model of surface reflectance. In the reported formulation, correspondence was established using a rank constraint, necessitating at least three viewpoints and three pairs of images. Here, it is revealed that the fundamental Helmholtz stereopsis constraint defines a nonlinear partial di...
متن کاملA Coupled Helmholtz Machine for PCA
In this letter we present a coupled Helmholtz machine for principal component analysis (PCA), where sub-machines are related through sharing some latent variables and associated weights. Then, we present a wake-sleep PCA algorithm for training the coupled Helmholtz machine, showing that the algorithm iteratively determines principal eigenvectors of a data covariance matrix without any rotationa...
متن کاملHelmholtz reciprocity in surface reconstruction: an overview
Reconstructing the 3D point mesh of an object from 2D images is a difficult problem. The task is even more complicated when the object has arbitrary and unknown surface reflectance. Traditional reconstructing techniques make explicit or implicit assumptions about the bidirectional reflectance distribution function (BRDF), which represents the surface reflectance. A widely used assumption is tha...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2016